Automated feature analysis of a structure

ABSTRACT

An automated structural feature and analysis system is disclosed. A 3D device emits a volume scanning 3D beam that scans a structure to generate 3D data that is associated with a distance between the 3D device and each end point of the 3D beam positioned on the structure. An imaging device captures an image of the structure to generate image data with the structure as depicted by the image of the structure. A controller fuses the 3D data of the structure generated by the 3D device with the image data of the structure generated by the imaging device to determine the distance between the 3D device and each end point of the 3D beam positioned on the structure and to determine a distance between each point on the image. The controller generates a sketch image of the structure that is displayed to the user.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation from U.S. nonprovisional applicationSer. No. 17/223,955, which claims the of benefit U.S. nonprovisionalapplication Ser. No. 16/453,168, which claims the benefit of U.S.Provisional Application No. 62/689,983, filed Jun. 26, 2018, all ofwhich are incorporated herein by reference in their entireties.

BACKGROUND

The invention generally relates to architectural analysis of structuresand specifically to three-dimensional (3D) modelling of the structure toexecute the architectural analysis.

Surveys of structures, such as roof surveys, are often times requiredfor different purposes. For example, roof surveys are required for theinstallation of solar panels to determine the amount, size, andpositioning of the solar panels based on the roof structure, roofinspections with regard to whether a roof upgrade is required forintegrity and regulatory compliance to determine the amount, size, andpositioning of roof materials, damage assessment of a roof to determinethe amount, size and positioning of roof materials required to repairthe damaged area, and so on. Typically, a surveyor is required tophysically mount themselves to the roof of the structure to assess theroof to generate an estimate of the amount, size, and positioning of thenecessary roof materials to adequately upgrade the roof whether that bewith solar panels, an upgraded roof and so on.

The conventional implementation of a surveyor physically mounted on theroof significantly increases the amount of time and cost associated withgenerating an estimate of the time and roof materials necessary toadequately upgrade the roof while decreasing the accuracy of theestimate. Each estimate requires an actual surveyor to evaluate the roofand to devote significant time to mount the roof and to generate anestimate resulting in a significant increase in cost. Further, thesurveyor is simply a person attempting to generate the estimate while ona roof rather than an automated system which decreases the accuracy ofthe estimate.

Further conventional automated roof surveying systems, such as UnmannedAerial Vehicles (UAV), simply capture images of the roof from an aerialperspective above the roof. Such conventional images generated from anaerial perspective skew the roof such that the edges of different planesof the roof may be difficult to discern as well as the angles of theplanes relative to the UAV as well as the structural features of theplanes due to the positioning of the UAV above the roof. Further,different obstructions included on the roof, such as chimneys andskylights, that should be excluded from an evaluation of the roof mayalso be difficult to discern from the UAV images. The difficulty indiscerning the roof hinders the accuracy in assessing the roof withregard to upgrading the roof not only by automated systems that evaluatethe images but also by humans. Automated systems and even humans havedifficulty in accurately discerning edges, angles, obstructions,features and so on of the roof resulting inaccurate estimates withregard to upgrading the roof.

SUMMARY OF THE INVENTION

Provided herein are systems, methods and compositions for a . . . .

The methods, systems, and apparatuses are set forth in part in thedescription which follows, and in part will be obvious from thedescription, or can be learned by practice of the methods, apparatuses,and systems. The advantages of the methods, apparatuses, and systemswill be realized and attained by means of the elements and combinationsparticularly pointed out in the appended claims. It is to be understoodthat both the foregoing general description and the following detaileddescription are exemplary and explanatory only and are not restrictiveof the methods, apparatuses, and systems, as claimed.

Accordingly, it is an object of the invention not to encompass withinthe invention any previously known product, process of making theproduct, or method of using the product such that Applicants reserve theright and hereby disclose a disclaimer of any previously known product,process, or method. It is further noted that the invention does notintend to encompass within the scope of the invention any product,process, or making of the product or method of using the product, whichdoes not meet the written description and enablement requirements of theUSPTO (35 U. S.C. § 112, first paragraph) or the EPO (Article 83 of theEPC), such that Applicants reserve the right and hereby disclose adisclaimer of any previously described product, process of making theproduct, or method of using the product. It may be advantageous in thepractice of the invention to be in compliance with Art. 53(c) EPC andRule 28(b) and (c) EPC. All rights to explicitly disclaim anyembodiments that are the subject of any granted patent(s) of applicantin the lineage of this application or in any other lineage or in anyprior filed application of any third party is explicitly reserved.Nothing herein is to be construed as a promise.

BRIEF DESCRIPTION OF THE DRAWINGS

In the accompanying figures, like elements are identified by likereference numerals among the several preferred embodiments of thepresent invention.

FIG. 1 illustrates side-view of a structure that may be assessed withregard to an architectural analysis of the structure according to anexemplary embodiment of the present disclosure;

FIG. 2 illustrates a top-elevational view of a structure that includes acomplicated roof with several different structural elements with severalof the structural elements having significant pitches such that theangles of the pitches slope significantly towards the groundsignificantly increasing the difficulties for the surveyor to maneuveraround the structure to adequately conduct the architectural assessmentaccording to an exemplary embodiment of the present disclosure;

FIG. 3 illustrates a side-view of a dual 3DThree-Dimensional dataacquisition device (3D device) and imaging device configuration suchthat a 3D device and an imaging device is mounted on a UAV to scan astructure to perform an architectural analysis on the structureaccording to an exemplary embodiment of the present disclosure;

FIG. 4 illustrates an automated structural feature analysisconfiguration that fuses the 3DThree-Dimensional (3D) data with theimage data to generate a sketch image of the structure that depicts eachof the structural elements as well as the associated structuraldimensions for each corresponding structural element according to anexemplary embodiment of the present disclosure;

FIG. 5 illustrates a side-view of a point cloud map of the structurebased on the fused 3D data and the image data that includes a pluralityof cloud points that are arranged to depict a 3D representation of thestructure according to an exemplary embodiment of the presentdisclosure;

FIG. 6 illustrates a side-view of a dual 3D device and imaging deviceconfiguration such that a 3D device and an imaging device is mounted ona UAV to scan a structure to perform an architectural analysis on thestructure according to an exemplary embodiment of the presentdisclosure; and

FIG. 7 illustrates a top-elevational view of a sketch image that depictsthe structure with respect to each of the segmented structural elementsand obstructions as well as providing the dimensions for each of thecorresponding structural elements according to an exemplary embodimentof the present disclosure.

DETAILED DESCRIPTION OF THE INVENTION

The following Detailed Description refers to accompanying drawings toillustrate exemplary embodiments consistent with the present disclosure.References in the Detailed Description to “one exemplary embodiment,” an“exemplary embodiment,” an “example exemplary embodiment,” etc.,indicate the exemplary embodiment described may include a particularfeature, structure, or characteristic, but every exemplary embodimentmay not necessarily include the particular feature, structure, orcharacteristic. Moreover, such phrases are not necessarily referring tothe same exemplary embodiment. Further, when a particular feature,structure, or characteristic may be described in connection with anexemplary embodiment, it is within the knowledge of those skilled in theart(s) to effect such feature, structure, or characteristic inconnection with other exemplary embodiments whether or not explicitlydescribed.

The exemplary embodiments described herein are provided for illustrativepurposes, and are not limiting. Other exemplary embodiments arepossible, and modifications may be made to the exemplary embodimentswithin the spirit and scope of the present disclosure. Therefore, theDetailed Description is not meant to limit the present disclosure.Rather, the scope of the present disclosure is defined only inaccordance with the following claims and their equivalents.

Embodiments of the present disclosure may be implemented in hardware,firmware, software, or any combination thereof. Embodiments of thepresent disclosure may also be implemented as instructions applied by amachine-readable medium, which may be read and executed by one or moreprocessors. A machine-readable medium may include any mechanism forstoring or transmitting information in a form readable by a machine(e.g., a computing device).

For example, a machine-readable medium may include read only memory(“ROM”), random access memory (“RAM”), magnetic disk storage media,optical storage media, flash memory devices, electrical optical,acoustical or other forms of propagated signals (e.g., carrier waves,infrared signals, digital signals, etc.), and others. Further firmware,software routines, and instructions may be described herein asperforming certain actions. However, it should be appreciated that suchdescriptions are merely for convenience and that such actions in factresult from computing devices, processors, controllers, or other devicesexecuting the firmware, software, routines, instructions, etc.

For purposes of this discussion, each of the various componentsdiscussed may be considered a module, and the term “module” shall beunderstood to include at least one software, firmware, and hardware(such as one or more circuit, microchip, or device, or any combinationthereof), and any combination thereof. In addition, it will beunderstood that each module may include one, or more than one, componentwithin an actual device, and each component that forms a part of thedescribed module may function either cooperatively or independently fromany other component forming a part of the module. Conversely, multiplemodules described herein may represent a single component within anactual device. Further, components within a module may be in a singledevice or distributed among multiple devices in a wired or wirelessmanner.

The following Detailed Description of the exemplary embodiments will sofully reveal the general nature of the present disclosure that otherscan, by applying knowledge of those skilled in the relevant art(s),readily modify and/or adapt for various applications such exemplaryembodiments, without undue experimentation, without departing from thespirit and scope of the present disclosure. Therefore, such adaptationsand modifications are intended to be within the meaning and plurality ofequivalents of the exemplary embodiments based upon the teaching andguidance presented herein. It is to be understood that the phraseologyor terminology herein for the purpose of description and not oflimitation, such that the terminology or phraseology of the presentspecification is to be interpreted by those skilled in the relevantart(s) in light of the teachings herein.

System Overview

FIG. 1 illustrates side-view of a structure that may be assessed withregard to an architectural analysis of the structure. A structure 100includes a plurality of structural elements that are specific to thestructure 100 and provide variations within the structure 100. In orderto properly assess the architectural analysis of the structure 100, eachof the different structural elements are to be accurately identified andalso assessed as individual structural elements and then pieced togetherto formulate an accurate architectural analysis of the overallstructure. An architectural analysis of the structure 100 is an analysisof the structure 100 that is performed to determine the health of thestructure 100 and/or to assess improvements and/or upgrades that are tobe made to the structure 100. For example, the structure 100 may be aroof 120 of a house 110 that the owner of the house 110 requests toassess the current health of the roof 120 of the house 110 and whetheran upgrade to the roof 120 is required as well as an accurate estimate.

The structure 100 may be any type of structure that includes structuralelements where a user has an interest to maintain the health of thestructure 100 and in doing so may upgrade the structure 100. Examples ofthe structure 100 include but are not limited to the roof of structures,buildings, houses, streets, bridges, parking garages, roads, highways,airports, mining operations, and/or any other type of structure thatincludes structural elements where a user has an interest to maintainthe health of the structure that will be apparent to those skilled inthe relevant art(s) without departing from the spirit and scope of thedisclosure. The types of upgrades of the structure 100 that may beincorporated into an architectural analysis of the structure 100 includebut are not limited to solar panel installation and maintenance, roofand roof maintenance industries, damage assessment and claim adjustmentof roof damages, inspection and enforcement of building codescompliance, land and site surveys, mapping and surveying containerports, road inspections, damage assessment and/or any other of upgradeof the structure that may be incorporated into an architectural analysisthat will be apparent to those skilled in the relevant art(s) withoutdeparting from the spirit and scope of the disclosure.

Structural elements included in the structure 100 are identifiablecharacteristics of the structure 100 that differ from other structuralelements included in the structure 100. For example, as shown in FIG. 1, the structure 100 includes a roof 120 positioned on a house 110.Different structural elements of the roof 120 include a first plane 130a and a second plane 130 b. A plane associated with a structure includesa segmented portion of the structure 100 that differs from othersegmented portions of the structure 100. For example, the roof 120 is aroof that is segmented into the first plane 130 a and the second plane130 b. The first plane 130 a extends from a ridge 180 of the roof 120and is angled from the ridge 180 at a pitch 160. The second plane 130 bextends from the ridge 180 of the roof 120 and is angled from the ridge180 at a pitch 160. The first plane 130 a and the second plane 130 b aresegmented and are angled at the pitch 160 to encourage water run-off andso on the for the roof 120 thus requiring that the first plane 130 a andthe second plane 130 b to be segmented from each other.

The ridge 180 of the roof 120 is the joint between two or more planes asthe two or more planes then extend from the ridge 180 at the specifiedpitch 160. The pitch 160 is a structural dimension of the first plane130 a and the second plane 130 b and is the angle of the associatedplane relative to the horizon. A structural dimension is a measurementand/or dimension associated with a structural element of the structure100 that is required to adequately assess and upgrade the structure 100.Other structural dimensions of a structure include the rise 140 of theplanes, the run 150 of the planes, the span 170 of the roof, volume ofthe structural elements, surface area of the structural elements,height, slope, pitch, inclination, clearance texture, and/or any otherstructural dimension associated with structural elements of thestructure 100 that will be apparent to those skilled in the relevantart(s) without departing from the spirit and scope of the disclosure.Structural elements of the structure also include but are not limited tothe hip, rake, gable, eave, rooftop valleys, ridges, edges, and/or anyother type of structural element included in the structure 100 thatprovides variations within the structure 100 and are identifiablecharacteristics of the structure 100 that will be apparent to thoseskilled in the relevant art(s) without departing from the spirit andscope of the disclosure.

In order to assess and then upgrade the structure 100, a survey of thestructure 100 is required to determine the different structural elementsincluded in the structure 100 as well as the structural dimensionscorresponding to the different structural elements that include but arenot limited to how the different structural elements included in thestructure 100 are oriented relative to each other, the dimensions of thestructural elements, the pitch 160 of the structural elements and so on.FIG. 1 depicts different structural elements that require distinctivemeasurements to adequately perform an architectural analysis of the roof120 as obtained via a survey. For example, FIG. 1 depicts the structuralelements of the first plane 130 a and the second plane 130 b that aresegmented from each other with both the first plane 130 a and the secondplane 130 b having the structural dimensions of rise 140, pitch 160, run150, and the span 170 with each being distinctive measurements requiredby the survey to adequately assess and then upgrade the structure 100.The identification of the different structural elements and thecorresponding structural dimensions are required by the survey toadequately assess the structural elements of the structure 100 and thetime and materials required to adequately upgrade the structure 100based on the assessment.

Conventional approaches to executing a survey to conduct anarchitectural analysis of the structure 100 require that the surveyoractually physically mount the structure 100 and then execute theassessment and architectural analysis of each of the structural elementsand the structural dimensions. The actual physical mounting of thestructure 100 by the surveyor significantly increases the amount of timerequired to conduct each survey as well as the costs in that there is asignificant time allotment associated with each surveyor mounting thestructure 100 as well as a significant cost allocation required for eachsurveyor to be positioned on the structure 100 to perform the survey.The amount of structures that require a survey is often timessignificant and requiring a surveyor to actually mount each structureand then execute the survey significantly increases the cost to performeach survey as well as the time required.

For example, a large roofing company and/or insurance company may havehundreds of roofs to assess at a given time. Simply requiring that asurveyor actually physically be present at each location and then tophysically mount each roof and then manually perform the survey couldrequire a couple of hours to complete. In order to satisfy thesignificant demand for surveys, such roofing companies and/or insurancecompanies would have to employ a significant staff of surveyors tosatisfy the demand with an increased cost due to the time required foreach surveyor to manually conduct the survey.

Further, the structures required to be surveyed by surveyors to performan architectural analysis on the structures may be complicated in thatthe structures include several different structural elements withdifferent structural dimensions as well as being dangerous for asurveyor to maneuver on without falling from the structure and causingsignificant bodily injury. FIG. 2 illustrates a top-elevational view ofa structure that includes a complicated roof 220 with several differentstructural elements with several of the structural elements havingsignificant pitches such that the angles of the pitches slopesignificantly towards the ground significantly increasing thedifficulties for the surveyor to maneuver around the structure toadequately conduct the architectural assessment. Incorporatingconventional approaches in which a surveyor actually mounts the roof 220of the house 210 to conduct the survey for the architectural analysissimply increases the risk of physical harm to the surveyor as well assignificantly increasing the cost of the survey due to the significantincrease in difficulty to conduct the survey by the surveyor.

However, conventional approaches where images of the structure 200 arecaptured from a position located above the structure 200 as depicted inFIG. 2 where the image captured of the structure 200 is atop-elevational view of the structure 200 also provide deficient resultswith regard to executing a survey to generate an accurate architecturalanalysis. Conventional approaches that capture top-elevational views ofthe structure 200, such as capturing grey scale images of the structure200 by an imaging device positioned on a UAV eliminates the requirementof a surveyor physically positioning themselves on the roof 220 of thestructure 200. However, such conventional approaches have a decreasedaccuracy regarding the identification and the assessment of thedifferent structural elements and the associated structural dimensionsincluded in the structure 200.

Although, conventional approaches that capture a top-elevational view ofthe structure 200 eliminate the requirement for a surveyor to actuallymount the roof 220 of the structure 200, such conventional approachesstill require that a user assess the images captured of thetop-elevational view of the structure 200 to identify the differentstructural components and the associated structural dimensions toexecute the architectural assessment of the structure 200. However, theimage that captures the top-elevational view of the structure 200 isoften times skewed based on the lack of contrast and distinction in thestructural elements that are depicted in the image that is required forthe user to adequately assess the image of the top-elevational view ofthe structure 200 to generate an accurate architectural analysis.

For example, as shown in FIG. 2 , structural elements included in theroof 220 of the structure 200 as depicted in the image of thetop-elevational view of the structure 200 may be difficult for a user toidentify. In such an example, the user may have difficulty indetermining the boundaries of the first plane 230 a and the second plane230 b as well as the ridge 280 separating the first plane 230 a form thesecond plane 230 b. Often times a conventional image of thetop-elevational view of the structure 200 may be skewed such that thepitch of the first plane 230 a and the second plane 230 b is difficultto discern such that the conventional image depicts to the user that thefirst plane 230 a and the second plane 230 b are actually a single planewithout a ridge 280 and are flat rather than having a pitch associatedwith each.

The conventional image of the top-elevational view of the structure 200may also be skewed such that the user may struggle to identify thelocation of the edges of the different structural elements of thestructure 200. In struggling to identify the location of the edges ofthe structural elements of the structure 200, the user may inaccuratelydetermine the structural dimensions of the structural elements of thestructure 200. The user in executing the survey via the conventionalimage of the top-elevational view of the structure 200 attempts todetermine the distances of the different structural dimensions of thestructural elements to determine the amount of material required toupgrade each of the structural elements included in the structure 200.The inability of the user to accurately identify the location of theedges of the structural elements due to the conventional image of the ofthe top-elevational view of the structure 200 being skewed prevents theuser from accurately measuring the distances of the structuraldimensions of the structural elements thereby generating inaccuratesurveys.

For example, the skewed image of the conventional top-elevational viewof the structure 200 prevents the user from accurately identifying theedges of first plane 230 a and the edges of the second plane 230 b andthe edges of the third plane 230 c. Without being able to accuratelyidentify the edges of the first plane 230 a, the second plane 230 b, andthe third plane 230 c, the user is also unable to accurately determinethe distances of each of the edges of the first plane 230 a, the secondplane 230 b, and the third plane 230 c. In failing to accuratelydetermine the distances of the edges of the first plane 230 a, thesecond plane 230 b, and the third plane 230 c, the user is also unableto accurately determine the amount of material required to adequatelyupgrade the first plane 230 a, the second plane 230 b, and the thirdplane 230 c.

Further, the skewed image of the conventional top-elevational view ofthe structure 200 also prevents the user from accurately identifying thedifferent structural elements. For example, the user struggles toidentify the valley 290 located between the third plane 230 c and thefourth plane 230 d. The conventional image of the top-elevational viewof the structure 200 being skewed causes the user to struggle inidentifying that the third plane 230 c and the fourth plane 230 d slopeat a pitch and are actually distinct planes separated by the valley 290.The skewed conventional image of the top-elevational view of thestructure 200 also causes the user in having difficulty in identifyingother structural elements included in the structure 200 such asidentifying the eve 240, the rake 250, the gable 260, and/or the hip270. Each of these different structural elements require differentmaterials to be adequately upgraded. The failure of the user toadequately identify each of these different structural elements due tothe conventional image of the top-elevational view of the structure 200being skewed results in inaccurate estimates as the appropriatematerials required to adequately upgrade each.

The failure of the conventional image of the top-elevational view of thestructure 200 to clearly depict the different structural elements andthe structural dimensions of the structure 200 results in the userinaccurately determining the amount of material required to upgrade thestructure 200 as well as the type of material and the different types ofstructural dimensions required to adequately upgrade the roof In failingto do so, the user generates inaccurate assessments regarding the costof materials, labor required, as well as the time to complete theupgrade of the structure 200 when generating the architectural analysisof the structure 200.

Rather than simply have a skewed image of the top-elevational view ofthe structure 200 for the user to inaccurately perform the architecturalanalysis of the structure 200, a 3D scan of the structure 200 may befused together with an image of the structure 200. The 3D scan of thestructure 200 may provide the distance of each end point of a 3D beampositioned on the structure 200. The image of the structure 200 mayprovide image data of the structure 200 that depicts each of thestructural elements of the structure 200. The fusing of the 3D scan ofthe structure 200 and the image of the structure 200 may provide thedistance between any two points positioned on the structure such thatthe distance of any structural dimension of any structural element ofthe structure 200 may be determined from the fusing of the 3D scan andthe image. A sketch image may then be generated from the fusing of the3D scan and the image that depicts each of the structural elements assegmented from the structure 200 with the appropriate structuraldimensions for each of the structural elements thereby enabling anaccurate survey of the structure 200 to be automatically generatedwithout requiring a surveyor to physically mount the structure 200 toconduct the survey.

FIG. 3 depicts a side-view of a dual 3D device and imaging deviceconfiguration such that a 3D device and an imaging device is mounted ona UAV to scan a structure to perform an architectural analysis on thestructure. The dual 3D device and imaging device configuration 300includes a UAV 350 that is positioned above a roof 320 of a structure310. The UAV 300 includes a 3D device that generates a volume scanning3D beam 340 that scans the roof 320 of the structure 310 to generate 3Ddata that is associated with a distance between the 3D device and eachend point of the 3D beam positioned on the structure. An imaging deviceis also mounted to the UAV 300 and captures a top-elevational image 360of the roof 320 of the structure 310 to generate image data associatedwith the roof 320 of the structure 310 as depicted by the image 360.

FIG. 3 depicts that the 3D device and the imaging device are mounted onthe UAV 350. However, the 3D device and the imaging device may bemounted on any type of object that enables the 3D device to adequatelyemit a volume scanning 3D beam to scan the structure 310 and the imagingdevice to adequately capture an image 360 of the structure 310 will beapparent to those skilled in the relevant art(s) without departing fromthe spirit and scope of the disclosure.

The 3D device is mounted to the UAV 350 and emits a volume scanning 3Dbeam 340 that scans the roof 320 of the structure 310 such that endpoint of the 3D beam 340 is positioned throughout the roof 320 of thestructure 310. In positioning the end point of the 3D beam 340throughout the roof 320 of the structure 310, the 3D device generates 3Ddata associated with a distance between the 3D device and each end pointof the 3D beam 340 positioned on the roof 320 of the structure 310. The3D data is then generated for each end point of the 3D beam 340positioned on the roof 320 of the structure 320 such that the distanceof each of the different structural elements of the roof 320 from the 3Ddevice 340 may be determined.

For example, the 3D device may be a 3D device that may emit the volumescanning 3D beam 340 to scan the first plane 330 a and the second plane330 b and the ridge 380 positioned on the roof 320. In scanning the 3Dbeam 340 across each of the structural elements included in the roof320, the 3D data that depicts the distance of each position on each ofthe structural elements from the 3D device may be determined such thateach of the points included in the first plane 330 a and the secondplane 330 b and the ridge 380 may have 3D data generated that depictsthe distance of each of the points from the 3D device. The 3D device maybe a 3D device that emits the 3D beam 340 that may be a 3D beam 340 suchthat the 3D beam 340 may be a volume scanning 3D beam. However, the 3Ddevice may also include other type of devices that emit 3D beams 340such as but not limited to a photogrammetry device that emits a 3D beam,a structured light 3D acquisition device that emits a 3D beam and/or anyother type of 3D device that generates a 3D beam such that the 3D datagenerated by the 3D beam 3D includes that is associated with a distancefrom the end point of the 3D beam 340 and each position on the structure310 may be incorporated will be apparent to those skilled in therelevant art(s) without departing from the spirit and scope of thedisclosure.

The imaging device is also mounted to the UAV 350 and captures an image360 of the roof 320 of the structure 310 to generate image dataassociated with the roof 320 of the structure 310 as depicted by theimage 360 of the roof 320 of the structure 310. The imaging device maybe a visible light imaging device, an infrared imaging device, anultraviolet imaging device, and/or any other type of imaging device thatadequately captures an image 360 of the structure 310 and depicts eachof the structural elements included in the structure 310 that will beapparent to those skilled in the relevant art(s) without departing fromthe spirit and scope of the disclosure. The image 360 captured by theimaging device may be a grey scale image, a color image, a still frameof the roof 320 of the structure 310, streaming video of the roof 320 ofthe structure and/or any other type of image that depicts each of thestructural elements included in the structure 310 that will be apparentto those skilled in the relevant art(s) without departing from thespirit and scope of the disclosure. The image data generated by theimage 360 captured by the imaging device is data associated with theroof 320 of the structure 310 that is then translated into each pixelincluded in the image 360 to accurately depict each of the structuralelement included in the roof 320.

The image 360 of the roof 320 of the structure 310 captured by theimaging device may depict the roof 320 such that each of the structuralelements included in the roof 320 may be accurately depicted by theimage 360. In doing so, each of the structural elements may beaccurately identified from the image 360 as well as each of the edgesand so on for each of the structural elements such that each of thestructural elements may be accurately identified and differentiated fromeach of the other structural elements. For example, the image 360 of theroof 320 may accurately depict the first plane 330 a and the secondplane 330 b as well as the ridge 380 such that each of the edgesassociated with the first plane 330 a and the second plane 330 b may beaccurately identified such that the first plane 330 a is differentiatedfrom the second plane 330 b along with the edges of the first plane 330a and the second plane 330 to be identified such that the divisionbetween the first plane and the second plane 330 b via the ridge 380 mayalso be accurately identified. In doing so, the image 360 may not beskewed and may depict each of the structural elements and their distinctfeatures so that each of the structural elements may be accuratelydistinguished from each of the other structural elements as well as theedges of the structural elements and so on.

Rather than simply having a single conventional image of thetop-elevational view of the roof 320 of the structure 310 that skews thestructural elements of the roof 320 such that the structural elements320 are difficult to depict from the conventional image, the dual 3D andimaging device configuration 300 incorporates both a 3D scan 340 of theroof 320 and an image 360 of the roof 320. The 3D scan 340 of the roofmay generate a three-dimensional (3D) model of the roof 320 that mayassist in the identification of the different structural elements of theroof 320 in addition to the image 360 of the roof 320.

However, the 3D data of the roof 320 as generated by the 3D device maybe fused together with the image data of the structure generated by theimaging device. The fusing together of the 3D data with the image datamay enable any point positioned on the roof 320 as depicted by the image360 to have a corresponding distance associated with the point asdetermined from the position of the 3D device and the corresponding endpoint of the 3D beam positioned on the structure at the correspondingpoint on the image 360. In doing so, a distance between each point inthe image 360 may be determined. In an embodiment, the 3D device and theimaging device may be positioned on a pan-tilt servo motor positioned onthe UAV may scan the roof 320 of the structure 310 to construct a pointcloud map of the surveyed structure 310.

FIG. 4 depicts an automated structural feature analysis configurationthat fuses the 3D data with the image data to generate a sketch image ofthe structure that depicts each of the structural elements as well asthe associated structural dimensions for each corresponding structuralelement. The automated structural feature analysis configuration 400includes a controller 410, a 3D device 420, an imaging device 430, auser interface 440, a structural element server 405, and a neuralnetwork 415. The controller 410 may fuse the 3D data 450 of thestructure 310 generated by the 3D device 420 with the image data 460 ofthe structure 310 generated by the imaging device 430 to determine thedistance between the 3D device 420 and each end point of the 3D beam 340positioned on the structure 310 and to determine a distance between eachpoint on the image 360.

The controller 410 may be a device that is capable of electronicallycommunicating with other devices. Examples of the controller 410 mayinclude a mobile telephone, a smartphone, a workstation, a portablecomputing device, other computing devices such as a laptop, or a desktopcomputer, cluster of computers, set-top box, and/or any other suitableelectronic device that will be apparent to those skilled in the relevantart(s) without departing from the spirit and scope of the disclosure.

The controller 410 may evaluate the image of the structure 310 ascaptured by the imaging device 430 to identify each of the differentstructural elements included in the structure 310 in executing thearchitectural analysis of the structure 310. Identifying of thestructural elements is required in executing the architectural analysisof the structure 310 such that the appropriate materials required toupgrade each of the structural elements are properly identified so thatan appropriate estimate of materials necessary to upgrade the structuralelements may be generated. In addition to identifying each of thedifferent structural elements included in the structure 310 to executethe architectural analysis of the structure 310, the structuraldimensions for each of the structural elements are also required suchthat the appropriate amounts of each material included in the structuralelements may be estimated in that that the dimensions of each structuralelement dictates the amount of materials required to upgrade thestructural element.

The controller 410 in fusing the 3D data with the image data may enablethe controller 410 to measure the structural dimensions of eachcorresponding structural element as depicted in the image generated bythe imaging device 430. As the controller 410 measures the distancebetween two points on the image 360 generated by the imaging device 430to determine the structural dimensions of the structural elements, thecorresponding 3D data 450 as generated by the 3D device 420 for thosetwo points may be fused with the image 360. In doing so, the distancebetween each point positioned on the image 360 and the 3D data 450 maybe automatically fused with the image 360 such the controller 410 mayautomatically determine the distance between each point positioned onthe image 360 and the 3D device 450. From the distances between eachpoint positioned on the image 360 and the 3D device 450, the controller410 may determine the distance between each of the two points selectedon the image 360 thereby generating the measurement of the distancebetween the two points.

For example in returning to FIG. 1 , the UAV 350 may capture the image360 of the roof 120. Due to the increased quality of the image 360, thecontroller 410 may identify the first plane 130 a, the second plane 130b, and the ridge 180 as distinct structural elements include in the roof120 in executing the architectural analysis. In addition to identifyingthe structural elements, the controller 410 may also determine thestructural dimensions of the structural elements. In doing so, thecontroller 410 may evaluate the image 360 of the roof 120 and select afirst point located on the ridge 180 and a second point located at theedge of the first plane 130 a to determine the measurement of thestructural dimension of the run 150 of the first plane 130 a. Inselecting the two points on the image 360 to determine the measurementof the run 150, the 3D data 450 associated with the two selected pointsis fused with the image 360 such that as the controller 410 selects thetwo points, the distance between the point at the ridge 180 and thepoint at the edge of the first plane 130 a is determined from the image360. In doing so, the controller 410 may then determine the distancebetween the point at the ridge 180 and the point on the edge of thefirst plane 130 a to determine the measurement of the run 150.

The identification of each of the structural elements of the structure310 as depicted by the image 360 and then the determination of thedistance of each structural dimension associated with each of thestructural elements based on the fusing of the 3D data 450 with theimage data 460 depicted in the image 360 may enable the controller 410to then generate a sketch image 480 of the structure 310. The controller410 may display the sketch image 480 to the user via the user interface440. The sketch image 480 depicts the structure 310 based on thedistance between the 3D device 420 and each point of the 3D beampositioned on the structure 310 and the distance between each pointdisplayed by the image 360. In doing so, the controller 410 may generatethe sketch image 480 of the structure 210 that identifies each of thestructural elements of the structure 310 as positioned relative to eachother on the sketch image 480 as well as providing the structuraldimensions of each of the structural elements and displaying thestructural dimensions to the user via the user interface 440 therebyenabling the user to easily comprehend the architectural analysis of thestructure 310. Thus, the sketch image 480 is a representation of thestructure 210 that identifies each of the structural elements of thestructure 210 as well as provides the corresponding structuraldimensions of each of the structural elements.

The fusing of the 3D data 450 as generated by the 3D device 420 and theimage data 460 as captured by the imaging device 430 may enable thecontroller 410 to identify each of the structural elements depicted bythe image 360 as well as determine the structural dimensions of each ofthe structural elements by simply analyzing the image 360. Thecontroller 410 does not have to move between an image 360 generated bythe imaging device 430 to identify structural elements and then a 3D mapgenerated by the 3D device 420 to determine the structural dimensions.Rather, the controller 410 may identify both the structural elements anddetermine the structural elements simply by analyzing the image 360 asfused with the 3D data 450 and the image data 460 thereby increasing theaccuracy and the efficiency in executing the architectural analysis ofthe structure 310 as opposed to a surveyor physically mounting thestructure 310 or a skewed image of the top-elevational view of thestructure 310. As a result, significant amounts of structures may havearchitectural analysis executed with decreased cost and increasedefficiency due to the efficient automation of the architectural analysisperformed on the image 360 with the 3D data 450 fused with the imagedata 460.

Fusing of 3D Data and Image Data

FIG. 5 is a side-view of a point cloud map of the structure based on thefused 3D data and the image data that includes a plurality of cloudpoints that are arranged to depict a 3D representation of the structure.As discussed above, the controller 410 may fuse the 3D data 450generated by the 3D device 420 with the image data 460 generated by theimaging device 430 such that the distance of each point positioned onthe image 360 of the structure 310 may be fused with the correspondinglocation of the point on the image 360 such that the distances betweenany two points included in the image 360 may be determined. In fusingthe 3D data 450 with the image data 460, the controller 410 may generatea point cloud map 500 of the structure 510 based on the fused 3D data450 and the image data 460.

As shown in FIG. 5 , the point cloud map 500 depicts a plurality ofcloud points that is arranged to depict a 3D representation of thestructure 510. Each of the cloud points included in the point cloud map500 may be generated based on the 3D beam 340 positioning itself on eachof the different positions on the structure 310 and generating 3D data450 for each of the cloud points. The 3D data 450 for each of the cloudpoints may include a distance from the position of the cloud point asthe 3D beam 340 is positioned at that particular point on the structure310 to the 3D device 420. For example, the 3D device 420 may generatethe 3D beam 340 and position the 3D beam 340 at the position representedby the cloud point 540 a and the position represented by the cloud point540 b. The 3D data 450 generated for the cloud point 540 a and the cloudpoint 540 b is the distance from the end point of the 3D beam 340positioned on the structure 510 at the cloud point 540 a and the cloudpoint 540 b to the 3D device 420. In doing so, the 3D data 450 includedwith the cloud point 540 a and the cloud point 540 b is the distancefrom the cloud point 540 a and the cloud point 540 b as positioned onthe structure 510 from the 3D device 420.

In addition to each cloud point including 3D data 450 that provides thedistance between each cloud point and the 3D device 420, each cloudpoint may also provide 3D data 450 that includes the distance betweeneach cloud point and each other cloud point positioned on the pointcloud map 500. As noted above, the controller 410 may determine thedistance between each point as depicted by the image 360 and each otherpoint depicted by the image 360 based on the fusion of the 3D data 450with the image data 460. The controller 410 may determine the distancebetween each point depicted by the image 360 and each other pointdepicted by the image 360 based on the distance between each pointpositioned on the structure 510 and the 3D device 420. As a result, the3D data 450 provided by each cloud point included in the point cloud map500 may also include the distance between each cloud point and eachother cloud point positioned on the point cloud map 500.

For example, the structural dimension of the ridge 580 may be determinedbased on the 3D data 450 provided by cloud point 540 a and cloud point540 b. The 3D data 450 provided by the cloud point 540 a and the cloudpoint 540 b may provide the distance between the cloud point 540 a andthe cloud point 540 b thereby providing the structural dimension of theridge 580. Thus, the point cloud map 500 provides the distance betweenthe 3D device and each cloud point included in the point cloud map 500and the distance between each cloud point included in the point cloudmap.

In addition to determining the distance of each structural elementincluded in the structure 510 as depicted by any two cloud pointspositioned on the point cloud map 500, the controller 410 may alsodetermine the structural dimension of pitch for each cloud pointincluded the point cloud map 500. The controller 410 may determine apitch of each plane 530 a and 530 b based on the distance between the 3Ddevice 420 and each end point of the 3D beam 340 positioned on thestructure 510 and the distance between each point on the image 360. Eachplane 530 a and 530 b included in the structure 510 is a portion of thestructure 510 that is positioned at a different pitch relative to eachother plane included in the structure 510. As noted above, the distancebetween any two points as depicted by the image 360 may be required todetermine the structural dimension of the distance of differentdimensions included in each of the structural elements in executing thearchitectural analysis of the structure 510. However, the pitch of eachstructural element is also required to adequately execute thearchitectural analysis of the structure 510 to adequately upgrade thestructure 510.

Each cloud point may also provide 3D data 450 that includes the pitch ofeach cloud point positioned on the point cloud map 500. As noted above,the controller 410 may determine the distance between each point asdepicted by the image 360 and each other point depicted by the image 360based on the fusion of the 3D data 450 with the image data 460. Thecontroller 410 may determine the pitch of each point depicted by theimage 360 based on the distance between each point positioned on thestructure 510 and the 3D device 420. As a result, the 3D data 450provided by each cloud point included in the point cloud map 500 mayalso include the pitch of each cloud point positioned on the point cloudmap 500.

For example, the structural dimension of the pitch for the first plane530 a and the second plane 530 b may be determined based on the 3D dataprovided by the cloud points positioned on the first plane 530 a and thesecond plane 530 b. The 3D data 450 provided by the cloud pointspositioned on the first plane 530 a and the second plane 530 b mayprovide the pitch of the first plane 530 a and the second plane 530 b.Thus, the controller 410 may determine the pitch of each plane 530 a and530 b included in the structure 510 based on the distance between the 3Ddevice 420 and each cloud point included in the point cloud map 500 andthe distance between each cloud point included in the point cloud map500. In an embodiment, each cloud point included in the point cloud map500 may be colored coded based on the distance of each cloud pointpositioned from the 3D device 420. The user may also rotate the 3D pointcloud map 500 of the structure 510 to assist in analyzing the structuralelements included in the structure 510.

As discussed above, the controller 410 may fuse together the 3D data 450as generated by the 3D device 420 with the image data 460 captured bythe imaging device 430. In doing so, the controller 410 may determinethe structural dimensions of each structural element included in thestructure 310 simply by selecting different points of the structure asdepicted by the image 360 of the structure 310. FIG. 6 depicts aside-view of a dual 3D and imaging device configuration such that a 3Ddevice and an imaging device is mounted on a UAV to scan a structure toperform an architectural analysis on the structure. In an embodiment,the 3D device 420 and the imaging device 430 may be mounted on a pantilt gimbal coupled to the UAV 350 and the gimbal may move the 3D device420 on the x-y axis such that the 3D beam 340 may scan the roof 320 ofthe structure 310 at each point on the roof 320 to capture the distanceof each point on the roof 320 from 3D device 420 positioned in thegimbal coupled to the UAV 350.

The 3D data 450 generated by the 3D device 420 and the image data 460captured by the imaging device 430 may be calibrated such that the 3Ddata 450 aligns with the image data 460. In doing so, each point of thestructure 310 as depicted by the image 360 may align with each cloudpoint of the structure 510 as depicted by the point cloud image 500. Asthe controller 410 selects a point on the structure 310 as depicted bythe image 360, the selected point may align with the corresponding cloudpoint of the structure as depicted by the point cloud image 500 and thusprovide the accurate structural dimensions for the point as depicted bythe image 360 without the controller 410 having to go between the image360 and the point cloud image 500. As a result, the controller 410 mayidentify a point positioned on the structure 310 as clearly depicted inthe image 360 and also determine the appropriate structural dimensionsfor the selected point depicted in the image 360 based on the 3D data450 for the selected point being aligned with the image data 460depicted by the image 360.

After the controller 410 aligns the 3D data 450 with the image data 460as depicted by the image 360 and the point cloud map 500, the controller410 may determine different structural dimensions based on the distancefrom a selected point on the structure 310 as depicted by the image 360and the 3D device 420 positioned on the UAV 350. For example, thecontroller 410 may select a point positioned on the edge 390 of thefirst plane 330 a as depicted by the image 360 of the structure 310 aswell as a point positioned on the ridge 380. Based on the 3D data 450being fused with the image data 460 by the controller 410, thecontroller 410 may determine the edge—drone distance 620 of the pointpositioned on the edge 390 from the 3D device 420 positioned on the UAV350 based on the 3D data 450 associated with the point positioned on theedge 390. The controller 410 may also determine the ridge—drone distance630 of the point positioned on the ridge 380 from the 3D device 420positioned on the UAV 350 based on the 3D data 450 associated with thepoint positioned on the ridge 380.

The 3D data 450 may also include the angle 650 between the 3D beam 340that is positioned at the point on the ridge 380 and the 3D beam 340that is positioned at the point on the edge 390. The controller 410 maythen based on trigonometric calculations determine the structuraldimensions of the distance between the point on the ridge 380 and thepoint on the ridge 390 which is the roof flap length L 640 as well asthe pitch 610 of the first plane 330 a. With the edge—drone distance620, the ridge—drone distance 630, and the angle 650 known by thecontroller 410 via the 3D data 450 for each of the respective points,the controller 410 may determine the pitch 610 of the first plane 330 abased on the sine of the edge—drone distance 620 and the ridge—dronedistance 630. The controller 410 may then determine the roof flap lengthL 640 based on the cosine of the pitch 610 and the edge—drone distance620. Thus, the controller 410 may determine different structuraldimensions associated with the corresponding structural elementsincluded in the structure 310 based on the distance of various pointspositioned on the structure 310 as depicted by the image 340 to the 3Ddevice 420 as determined from the 3D data 450 that is fused with theimage data 460.

Automated Segmentation

As noted above, each of the structural elements included in thestructure 200 may vary significantly from each other as well as beingcustomized with regard to the structure 200. Each structure 200 may havecustomized structural elements that differ from similar structuralelements that are included in other structures. For example, FIG. 2depicts a complicated roof 220 that is positioned on the house 210.There are limited structural elements that are actually symmetricstructural dimensions to each other. In such an example, the fourthplane 230 d and the fifth plane 230 e actually include symmetricstructural dimensions to each other. However, the first plane 230 a, thesecond plane 230 b, and the third plane 230 c each have structuraldimensions that differ significantly from each other due to thepositioning of the dormer 205, the roof vent 215 and so on.

Thus, the controller 410 may segment each of the structural elementsincluded in the structure 200 to account for the customization of eachof the structural elements regardless as to whether the structuralelement has a cut-out included in the structural element to circumvent aroof vent 215, having customized structural dimensions, and so on. Thesegmenting of each of the structural elements includes theidentification and/or tracing of the boundaries of the of the structuralelements to delineate each of the structural elements from each of theother structural elements based on the boundaries of each of thecorresponding structural elements. For example, the controller 410 maysegment the first plane 230 a by identifying and/or tracing theboundaries of the first plane 230 a. In doing so, the controller 410 maydelineate the first plane 230 a from the second plane 230 b, the thirdplane 230 c, as well as the chimney 225 that is inserted through thefirst plane 230 a such that the first plane 230 a includes a cut-out toaccount for the chimney 225.

he controller 410 segments each plane included in the structure 200 asdepicted by the image 360 of the structure 200 so that each plane issegmented from each other plane included in the structure as depicted bythe image 360 based on the fused 3D data 450 and the image data 460depicted in the point cloud map 500. Each point depicted by the image360 corresponds to a cloud point on the point cloud map 500 thatprovides the distance between the 3D device 420 and each cloud point andthe distance between each cloud point.

As discussed above, the 3D data 450 may be fused together with the imagedata 460 such that the image 360 of the structure 200 may depict each ofthe structural elements via the image data 460 but each point includedin the structural elements may have corresponding 3D data 450 due to thefusion of the 3D data 450 with the image data 460. In doing so, thedistance between each point as depicted by the image 360 and the 3Ddevice 420 may be determined based on the image 360. Thus, thecontroller 410 may analyze the image 360 and segment each structuralelement included in the structure 200 as depicted by the image 360 toadequately differentiate each structural element from each of the otherstructural elements included in the structure 200.

The controller 410 determines a plurality of structural dimensionsassociated with each plane and a pitch associated with each plane and aplurality of distances between each plane and each other plane based onthe segmentation of each plane included in the structure as depicted bythe image 360 of the structure 200. As discussed above, each of thestructural dimensions associated with each structural element includedin the structure 200 may be determined by the controller 410 based onthe fusion of the 3D data 450 with the image data 460 due to thedistance between each point as depicted by the image 360 and the 3Ddevice 420. Each of the structural dimensions such as the distancebetween two points included in the first plane 230 a to determine thespan of the first plane 230 a may be determined by the controller 410simply by identifying and/or tracing the edge on the first plane 230 abetween the appropriate two points include on the edge of the firstplane 230 a. Thus, the controller 410 may determine the variousstructural dimensions of each of the structural elements by identifyingand/or tracing the structural elements as depicted by the image 360.

In determining the structural dimensions of each structural elementincluded in the structure 200 based on the segmentation of eachstructural element by the controller 410, the controller 410 maydetermine the customized amounts of material that is required toadequately upgrade each of the structural elements. Each customizedstructural element may have unique structural dimensions that are uniqueto that particular customized structural element. The segmenting of thecustomized structural element and then determining the customizedstructural dimensions for the customized structural element based on thesegmentation, may enable the controller 410 to accurately assess thematerials required to adequately upgrade the customized structuralelement. For example, the first plane 230 a appears to extend the lengthof the roof 220 of the house 210. However, the chimney 225 is includedin the first plane 230 a such that the first plane 230 a includes acut-out to account for the chimney 225. Simply assessing the structuraldimensions of the first plane 230 a without accounting for the cut-outfor the chimney 225 may result in an increased order of materials toadequately upgrade the first plane 230 a that is unnecessary due to thecut-out included in the first plane 230 a to account for the chimney225.

The controller 410 locates each structural element that formulates eachplane included in the structure as depicted by the image 360 asdesignated by each line that encompasses each plane. In segmenting eachstructural element, the controller 410 may locate each structuralelement by tracing each structural element with a line that encompasseseach structural element. In tracing each structural element, thecontroller 410 may trace the appropriate features of the structuralelement such that the controller 410 may determine the appropriatestructural dimensions for each structural element. For example, thecontroller 410 may trace the boundaries of the second plane 230 b todifferentiate the second plane 230 b from the first plane 230 a and thethird plane 230 c. In tracing the boundaries of the second plane 230 b,the controller 410 may determine the appropriate structural dimensionsof the second plane 230 b to determine the amount of material requiredto adequately upgrade the second plane 230 b.

The controller 410 may identify each structural element that formulateseach plane included in the structure 200 to identify a structuralfeature of the structural element of each plane.

The structural feature is a feature provided by each element to thestructure as formulated in each corresponding plane. As noted above, thecontroller 410 may locate each structural element included in thestructure 200 by tracing the boundaries of the structural element todifferentiate the structural element from the other structural elementsincluded in the structure 200. In addition to locating, the controller410 may also identify each structural feature included in eachstructural element. Each structural element may include structuralfeatures. For example, the first plane 230 a includes the ridge 280 as afirst boundary and then an edge 235 as a second boundary while the sixthplane 230 f includes a first boundary of an eve 245. In locating each ofthe boundaries of each structural element via tracing each of theboundaries, the controller 410 may also identify each of the structuralfeatures that acts as the different boundaries for the structuralelement. In such an example, the controller 410 may identify aftertracing the first boundary of the first plane 230 a that the firstboundary is the ridge 280 while the second boundary is the edge 235 andthe first boundary of the sixth plane 230 f as the eve 245.

In identifying the structural feature represented by each of theboundaries included in the structural element, the controller 410 maydetermine the appropriate materials required to adequately upgrade eachof the structural features. For example, the material required toupgrade the ridge 280 as the first boundary of the first plane 230 a maydiffer from the material required to upgrade the edge 235 of the secondboundary of the first plane 230 a while the material required to upgradethe eve 245 of the first boundary of the sixth plane 230 f may alsodiffer. Thus, the controller 410 may determine the appropriate materialsrequired to adequately upgrade each of the different structural featuresrepresented by each of the boundaries included in the structural elementbased on the identification of each of the structural features by thecontroller 410.

The controller 410 may determine a distance of each element thatformulates each plane included in the structure 200. The distance is alength dimension associated with each structural element. As notedabove, the controller 410 may locate each structural element included inthe structure 200 by tracing the boundaries of the structural element aswell as identifying each of the structural features included in thestructural element. In addition to locating and identifying, thecontroller 410 may also determine the distance of each traced boundaryincluded in the structural element. As discussed in detail above, thecontroller 410 may determine the distance between any two pointsincluded in the image 360 due to the fusing of the 3D data 450 with theimage data 460.

In doing so, the controller 410 may simply trace the appropriateboundary of the structural element from a first point to a second pointand then determine the distance of the appropriate boundary based on thedistance of the first point and the second point from the 3D device 420due to the 3D data 450 being fused with the image data 460 as depictedby the image 360. For example, the controller 410 may simply trace theridge 280 of the first plane 230 a from a first point to a second pointon the image 360 and may determine the distance of the ridge 280 basedon the trace on the image 360. Thus, the controller 410 may locate,identify, and determine the distance of each boundary included in eachstructural element simply by interfacing with the image 360.

In addition to segmenting each structural element included in thestructure 200, the controller 410 may also segment each obstructionpositioned on the structure 200 as depicted by the image 360 of thestructure 200 so that each obstruction is segmented from each planeincluded in the structure 200 as depicted by the image 360. Eachobstruction is an obstruction positioned on the structure that is notincluded in any of the planes included in the structure 200. As notedabove, each structural element may be customized not only with regard tothe structural dimensions included in each structural element but alsoto account for obstructions that are positioned on the structuralelements. An obstruction is not included in the structural element butis rather positioned on the structural element and is accounted for bythe controller 410. Typically, the structural element may include acut-out to account for the obstruction positioned on the structuralelement such that the structural element may require less material dueto the cut-out to account for the obstruction as compared to if noobstruction is positioned on the structural element.

For example, the first plane 230 a includes the obstruction of thechimney 225 such that the first plane 230 a includes a cut-out toaccommodate the chimney 225 such that the first plane 230 a may requireless material due to the cut-out to accommodate the chimney 225 ascompared to if the chimney 225 was not positioned on the first plane 230a. In another example, the second plane 230 b includes the obstructionof the roof vent 215 and the dormer 205 such that the second plane 230 bincludes a cut-out to accommodate the roof vent 215 and the dormer 205such that the second plane 230 b may require less material due to thecut-out to accommodate the roof vent 215 and the dormer 205. Thus, thecontroller 410 may segment each obstruction to delineate eachobstruction from the structural element that the obstruction ispositioned on.

The controller 410 may locate each obstruction that is positioned on thestructure 200 as depicted by the image 360 as designated by each linethat encompasses each obstruction. Similarly to how the controller 410may locate each structural element by tracing the boundaries of thestructural element as depicted by the image 360, the controller 410 mayalso trace the boundaries of each obstruction as depicted by the image360 to delineate the obstruction from the structural element that theobstruction is positioned on. For example, the controller 410 may tracethe boundaries of the obstruction of the skylight 255 as positioned onthe seventh plane 230 g to delineate the skylight 255 from the seventhplane 230 g that the skylight is positioned on.

The controller 410 may also identify each obstruction that is positionedon the structure 200 to identify a structural feature that is a featureprovided by each obstruction to the structure. Similarly to how thecontroller 410 may also identify each structural feature included in thestructural element in addition to locating the structural feature, thecontroller 410 may also identify each obstruction that has been locatedby the controller 410. For example, the controller 410 may identify theobstruction of the skylight 255 as being a skylight 255 after thecontroller 410 traces the skylight 255 to delineate the skylight 255from the seventh plane 270 g. The controller 410 may also determine thedimensions of each located obstruction in a similar manner as thecontroller 410 determines the structural dimensions of the structuralelements as discussed in detail above. Obstructions positioned on thestructure 200 may include but are not limited to a roof vent, a chimney,a dormer, a skylight, a plumbing vent, a ridge vent, a gutter, and/orany other obstruction that may be positioned on the structure but is nota structural element that will be apparent to those skilled in therelevant art(s) without departing from the spirit and scope of thedisclosure.

The controller 410 may also incorporate algorithms to improve thequality of the image 360 of the structure 200 to further assist inlocating different structural elements of the structure 200 andobstructions. The controller 410 may incorporate image processingalgorithms that include but are not limited to a blob detectoralgorithm, corner detector algorithm, edge detector algorithm, contourfitting algorithm to recognize and delineate boundaries of thestructural elements as well as obstructions and/or damaged areas of thestructure and/or any other type of image processing algorithm that willbe apparent to those skilled in the relevant art(s) without departingfrom the spirit and scope of the disclosure.

In an embodiment, the user may segment each structural element and eachobstruction as the user engages the image 360 of the structure 200 asdisplayed by the user interface 440. In performing the segmentation, theuser may generate segmentation data 470 that is generated as the userengages the image 360 of the structure via the user interface 440. Thecontroller 410 may then incorporate the segmentation data 470 into thesegmentation process in segmenting each of the structural elements andthe obstructions in generating the structural dimensions for each of thestructural elements and the obstructions.

After each of the structural elements and the obstructions in thestructure 200 have been segmented, the controller 410 may generate thesketch image of the structure that is displayed to the user that depictseach segmented plane and each segmented obstruction included in thestructure 200 based on the segmentation of each plane and eachobstruction relative to the distance between the 3D device 420 and eachpoint of the 3D beam 340 positioned on the structure 200, the distancebetween each point on the image 360, and pitch of each plane. FIG. 7illustrates a top-elevational view of a sketch image 700 that depictsthe structure with respect to each of the segmented structural elementsand obstructions as well as providing the dimensions for each of thecorresponding structural elements.

As noted above, an architectural analysis of the structure 200 isrequired to determine the appropriate estimate as to the time,materials, and the overall scope of the work required to adequatelyupgrade the structure 200. The sketch image 700 of the structuretransforms the structure 200 to a depiction of the image relative toeach structural element such that the user may easily identify each ofthe structural elements included in the structure 200 rather than havingto decipher each of the structural elements as depicted by image 360 ofthe structure 200 and/or the point cloud image 500.

The user may easily identify and assess each structural element asdepicted by the sketch image 700. For example, the user may easilyidentify each of the structural elements 720(a-n) as depicted in thesketch image 700. Rather than having to decipher each of the structuralelements as depicted in the image 360 of the structure 200, the user mayeasily identify and assess each of the structural elements 720(a-n)easily depicted by the sketch image 700 without any additional detail,data, obstructions, shadows, shading, and so on that may be depicted bythe image 360 of the structure 200.

In addition to easily depicting each structural element of the structure200 for the user, the sketch image 700 may also depict each of thecorresponding structural dimensions for each of the structural elementssuch that the user may easily identify the structural dimensions of eachof the structural elements. For example, the user may easily identifyeach of the structural dimensions 710(a-n) as depicted in the sketchimage 700. Rather than having to determine the structural dimensionsthemselves and/or determine the structural dimensions from the pointcloud map 500 and so on, the user may easily identify and assess each ofthe structural dimensions 710(a-n) as easily depicted by the sketchimage 700 such that each structural dimension 710(a-n) is clearlyassociated with each corresponding structural element. In doing so, theuser may easily generate an accurate estimate as to the amount of timeand material required to upgrade each of the structural elementsincluded in the structure 200 thereby providing an efficient andaccurate architectural analysis of the structure 200 with a decrease incost.

In an embodiment, the controller 410 may import the sketch image 700into a more robust computer-aided design (CAD) package that may enablethe user to further analyze the sketch image 700 that includes thestructural elements 720(a-n) and the structural dimensions 710(a-n). Inmore complicated applications, such as generating an architecturalassessment on a mining operation, the details of the structural elements720(a-n) as well as the structural dimensions 710(a- n) may be immenseas well as minute. Importing the sketch image 700 into a more robust CADpackage, such as AutoCAD, the user may also incorporate the analysistools provided by the more robust CAD package in analyzing each of theseveral structural elements 720(a-n) as well as the structuraldimensions 710(a-n).

For example, the controller 410 may import the sketch image 700 of acomplicated bridge that is to be upgraded and includes an immense amountof structural elements 720(a-n) and corresponding structural dimensions710(a-n) into SolidWorks. The user may then incorporate the robust toolsincluded in SolidWorks to analyze each of the minute structural elements720(a-n) and corresponding structural dimensions 710(a-n). Thecontroller 410 may import the sketch image 700 in CAD packages thatinclude but are not limited to AutoCAD, SolidWorks and/or any other typeof CAD system that may assist in analyzing the sketch image 700 thatwill be apparent to those skilled in the relevant art(s) withoutdeparting from the spirit and scope of the disclosure.

In such an embodiment, the controller 410 may map each of the pixelsincluded in the sketch image 700 to a corresponding pixel included inthe CAD package. In doing so, the fused 3D data 450 and image data 460that is incorporated into each pixel of the sketch image 700 may bemapped to a corresponding pixel included in the CAD package such thatthe CAD package may continue to analyze each pixel that includes thefused 3D data 450 and the image data 460. Thus, the display of thesketch image 700 by the CAD package may be seamless to the user whileenabling the user to incorporate each of the tools included in the CADpackage to further analyze the structural elements 720(a-n) and thecorresponding structural dimensions 710(a-n) depicted by the sketchimage 700

Neural Network

Returning to FIG. 4 , the neural network 415 may assist the controller410 in identifying each of the structural elements and the obstructionsas well as the corresponding structural dimensions for the structure200. As noted above, the controller 410 may segment each of thestructural elements and obstructions. In doing so, the controller 410may locate each structural element and obstruction by tracing theboundaries of the structural element and the obstruction. The controller410 may identify each structural element and obstruction. The controller410 may also determine the appropriate structural dimensions for thestructural element and obstruction. Each time that the controller 410locates, identifies, and determines the structural dimensions for thestructural element and obstruction, the controller 410 may provide thisdata as learning data 495 and provide the learning data 495 to thestructural element server 405.

The neural network 415 may then apply a neural network algorithm such asbut not limited to a multilayer perceptron (MLP), a restricted BoltzmannMachine (RBM), a convolution neural network (CNN), and/or any otherneural network algorithm that will be apparent to those skilled in therelevant art(s) without departing from the spirit and scope of thedisclosure. Each time that a data set of learning data 495 that locatesthe particular structural element and/or obstruction, the identifying ofthe particular structural element and/or obstruction, and theappropriate structural dimensions for the particular structural elementand/or obstruction, the neural network 415 may continue to accumulateeach received data set of learning data 495 to further improve theaccuracy of the controller 410 in segmenting each of the structuralelements and/or obstructions included in the structure 200. The neuralnetwork 415 may provide the improved location and identification of eachstructural element and/or obstruction via the segmented data 455 to thecontroller 410 such that the controller 410 may continue to learn uponeach structural element and/or obstruction that is located, identified,and measured.

As the neural network 415 learns with each data set of learning data 495that includes the location, identification, and the structuraldimensions of the particular structural element and/or obstruction, theneural network 415 may assist the controller 410 such that thecontroller 410 in segmenting each structural element and/or obstructionmay eventually mimic the success of a human actually segmenting eachstructural element and/or obstruction. The fusion of the 3D data 450 andthe image data 460 may provide an additional layer of information forthe neural network 415 to learn from in that as the controller 410segments each structural element and/or obstruction as depicted by theimage 360, the learning data 495 provided to the neural network includesthe appropriate structural dimensions for the segmented structuralelement and/or obstruction. Rather than simply providing the locationand identification information as determined from the image data 460provided by the image 360, the appropriate structural dimensions arealso provided from segmenting the structural elements and/orobstructions via the image 360 based on the 3D data 450 fused with theimage data 460 in the image 360. Thus, the accuracy as well as the ratein which the neural network 415 continues to learn and recognize inassisting the controller 410 in segmentation may also increase.

For example, examining the image 360 with regard to the top-elevationalview of the structure 200, often times different shapes of thestructural elements, different shades of the structural elements,different textures of the structural elements, and/or any convolutionsin the structural elements may increase the difficulty in accuratelysegmenting each of the structural elements. Returning to FIG. 2 thatdepicts a complicated structure 200 with regard to a complicated roof220, segmenting the plane 230 f based on evaluating the image 360 of thestructure 200 may be difficult due to different shadows and/or shadingthat may be incorporated onto the roof 220 based on the various planesand other structural elements as well as obstructions.

In addition to segmenting the plane 2301, locating and identifying theeve 245 may also be difficult as due to the different shadows and/orshading, the eve 245 may look like an edge rather than eve 245 therebypreventing the eve 245 from being accurately located and identified.Over time with an increase in the sample size depicted by the learningdata 495 and deciphered by the neural network 415, the neural network415 may increase the accuracy in the controller 410 adequatelysegmenting the plane 230 f as well as accurately locating andidentifying the eve 245 regardless of the shadows and/or shadingdepicted by the image 360 of the structure 220 by providing segmenteddata 455 to the controller 410 that is of increased accuracy. Thesegmented data 455 may be of increased accuracy due to the increase insample size of learning data 495 that depicts various locations,identifications, and structural dimensions for various structuralelements and obstructions captured from various structures with variousdifferent settings such as shading and/or shadows. Thus, with each setof learning data 495 provided to the neural network 415, the accuracy ofthe segmented data 455 provided to the controller 410 by the neuralnetwork 415 to assist the controller 410 in accurately segmenting thestructural elements and obstructions continues to improve.

In an embodiment, in addition to the location and identification that isdetermined from the image data 460, color information may also bedetermined from the image data 460. Each image captured by the imagedevice 430 may also include color information in that each portion ofthe image captured by the image device 430 includes different shades ofcolor. In doing so, the different structural elements and/orobstructions captured by the image 360 may have different shades ofcolor resulting in different color information that is captured by theimage 360 and is included in the image data 460. The different colorinformation that is included in the image data 460 may provideadditional insight as to the type of structural element and/orobstruction that is associated with the different color information. Forexample, eve 245 may have increased shading as depicted in the capturedimage 360 while other structural elements and/or obstructions may havedecreased shading. Thus, the color information associated with eachstructural element and/or obstruction and included in the image data mayprovide additional insight as to the identification of the correspondingstructural element and/or obstruction.

In doing so, the controller 410 may incorporate the different colorinformation included in the image data 460 into the learning data 495that is provided to the structural element server 405. The controller410 may then provide the learning data 495 to the structural elementserver 405 that not only includes the location, identification, and thestructural dimensions of the particular structural element and/orobstruction but the color information as well. The addition of the colorinformation to the location, identification, and the structuraldimensions may further enhance the neural network 415 in assisting thecontroller 410 such that the controller 410 in segmenting eachstructural element and/or obstruction may eventually mimic the successof a human actually segmenting each structural element and/orobstruction. The fusion between the 3D data 450 and the image data 460that includes the color information may further improve and provide anadditional layer of information for the neural network 415 to learn fromin that as the controller 410 segments each structural element and/orobstruction as depicted by the image 360, the learning data 495 providedto the neural network 415 includes the appropriate structural dimensionsfor the segmented element and/or obstruction.

For example in an embodiment, the different segments may be annotatedvia color such that each segment is portrayed to the neural network 415as being associated with a specified color. Rather than simply providingthe location and identification information as determined from the imagedata 460 provided by the image 360, a specified color is also annotatedto each segment such and provided to the neural network 415 in thesegmenting of the structural elements and/or obstruction via the image360 based on the 3D data 450 fused with the image data 460 in the image360. Thus, the accuracy as well as the rate in which the neural network415 continues to learn and recognize in assisting the controller 410 insegmentation may also increase.

In an embodiment, the controller 410 may fuse the 3D data 450 generatedby the 3D device 420 with the image data 460 generated by the imagingdevice 430 such that the distance of each point positioned on the image360 of the structure 310 may be fused with the corresponding location ofthe point on the image 360 such that the distances between any twopoints may be determined. In addition to the distance of each pointpositioned on the image 360 of the structure fused with thecorresponding location of the point on the image, the color informationmay also be fused with the corresponding location of the point on theimage. In doing so, each point on the image 360 associated with thepoint as well as a color associated with the point. In fusing 3D data450 with the image data 460 that includes the color information inaddition to the distance for each point, the controller 410 may generatethe point cloud map 500 of the structure 510 based on the fused 3D data450 and the image data 460 that includes both the distance and the colorinformation.

In an embodiment, each of the different layers of information includedin the learning data 495 provided by the controller 495 to the neuralnetwork 465 may include the color information associated with each pixelbounded by a segment boundary that has been segmented. The differentlayers of information included in the learning data 495 provided by thecontroller 495 to the neural network 415 may also include annotationand/or label information associated with the segment. The differentlayers of information included in the learning data 495 provided by thecontroller 495 to the neural network 465 may also include the distanceinformation associated with each cloud point 540(a-n) from the pointcloud 500 that is included in the segment and is fused to the locationof each corresponding pixel included in the image 360 and the color ofeach corresponding pixel included in the image 360. Additional datacaptured from devices other than the 3D device 420 and the imagingdevice 450, such as thermal imaging and/or radar data, may also beincluded in the learning data 495 provided by the controller 495 to theneural network 415 to account for any water damage, humidity level, heatleakage, structural damage, and so on.

In an embodiment, the 3D device 420 and the imaging device 430 may bemounted to the UAV 350 to incorporate the 3D data 420 and the image data460 into search and rescue operations, structural and architecturalanalysis, utility monitoring, damage reports, environmental exposure,gas levels, and so on. The 3D device 420 and the imaging device 430mounted to the UAV 350 may analyze telemetry data and/or geolocationdata that is tagged with sensor data from remote nodes to aid withsearch and rescue operations and/or wildlife monitoring and/or disasterrelief and/or restoration operations. The mounting of the 3D device 420and the imaging device 460 onto UAV 350 may provide additional benefitsin widening the search area, achieving longer communication rangers,and/or avoiding obstacles in urban and/or suburban communities.Increased communication ranges may be achieved when UAV 350 are flyingat increased altitudes above sea level to provide increasedcommunication ranges than achieved at sea level due to the curvature ofthe earth. Such applications may include search and rescue in Man OverBoard scenarios at sea, locating persons and/or objects in remote areassuch as mountain or forest terrains, loan work monitoring and/orcommunications.

In an embodiment, the 3D device 420 and the imaging device 430 may bemounted on the UAV 350 that communicates over the sub-GHz range ofwireless communication. The 3D device 420 and the imaging device 430 maybe equipped with a geolocation transceiver to acquire and/or send thelocation of the 3D device 420 and the imaging device 430, The 3D device420 and the imaging device 430 may draw power from the UAV batteryand/or be equipped with their own batter and/or be solar powered. Thecontroller 410 may store data for asynchronous retrieval by the operatorand/or may be equipped with a satellite and/or communication transceiverto relay sensor and/or telemetry data in real-time. The data streams maybe geo tagged. The 3D device 420 and the imaging device 430 may includeadditional sensors such as cameras, radiation sensors, air qualitysensors, three-dimensional acquisition devices and so on.

CONCLUSION

It is to be appreciated that the Detailed Description section, and notthe Abstract section, is intended to be used to interpret the claims.The Abstract section may set forth one or more, but not all exemplaryembodiments, of the present disclosure, and thus, is not intended tolimit the present disclosure and the appended claims in any way.

The present disclosure has been described above with the aid offunctional building blocks illustrating the implementation of specifiedfunctions and relationships thereof. The boundaries of these functionalbuilding blocks have been arbitrarily defined herein for the convenienceof the description. Alternate boundaries may be defined so long as thespecified functions and relationships thereof are appropriatelyperformed.

What is claimed is:
 1. An automated structural feature analysis system,comprising: a Three-Dimensional (3D) device configured to emit a volumescanning 3D beam that scans a structure to generate 3D data that isassociated with a distance between the 3D device and each end point ofthe 3D beam positioned on the structure, an imaging device configured tocapture an image of the structure to generate image data associated withthe structure as depicted by the image of the structure; and acontroller configured to: fuse the 3D data of the structure generated bythe 3D device with the image data of the structure generated by theimaging device to determine the distance between the 3D device and eachend point of the 3D beam positioned on the structure and to determine adistance between each point on the image, and generate a sketch image ofthe structure that is displayed to the user that depicts the structurebased on the distance between the 3D device.
 2. The automated structuralfeature analysis of claim 1, wherein the controller is furtherconfigured to: determine each point of the 3D beam positioned on thestructure and the distance between each point on the image, determine apitch of each plane included in the structure based on the distancebetween the 3D device and each end point of the 3D beam positioned onthe structure and the distance between each point on the image, whereineach plane included in the structure is a portion of the structure thatis positioned at a different pitch relative to each other plane includedin the structure.
 3. The automated structural feature analysis system ofclaim 1, wherein the controller is further configured to: generate apoint cloud map of the structure based on the fused 3D data and theimage data that includes a plurality of cloud points that is arranged todepict a three-dimensional (3D) representation of the structure, whereinthe point cloud map provides the distance between the 3D device and eachcloud point included in the point cloud map and the distance betweeneach cloud point included in the point cloud map.
 4. The automatedstructural feature analysis system of claim 2, wherein the controller isfurther configured to determine the pitch of each plane included in thestructure based on the distance between the 3D device and each cloudpoint included in the point cloud map and the distance between eachcloud point included in the point cloud map.
 5. The automated structuralfeature analysis system of claim 3, wherein the controller is furtherconfigured to: segment each plane included in the structure as depictedby the image of the structure so that each plane is segmented from eachother plane included in the structure as depicted by the image based onthe fused 3D data and image data depicted in the point cloud map,wherein each point depicted by the image corresponds to a cloud point onthe point cloud map that provides the distance between the 3D device andeach cloud point and the distance between each cloud point.
 6. Theautomated structural feature analysis system of claim 4, wherein thecontroller is further configured to: determine a plurality of dimensionsassociated with each plane and a pitch associated with each plane and aplurality of distances between each plane and each other plane based onthe segmentation of each plane included in the structure as depicted bythe image of the structure.
 7. The automated structural feature analysissystem of claim 5, wherein the controller is further configured to:locate each element that formulates each plane included in the structureas depicted by the image as designated by each line that encompasseseach plane; identify each element that formulates each plane included inthe structure to identify a structural feature of each element of eachplane, wherein the structural feature is a feature provided by eachelement to the structure as formulated in each corresponding plane; anddetermine a distance of each element that formulates each plane includedin the structure, wherein the distance is a length dimension associatedwith each element.
 8. The automated structural feature analysis systemof claim 6, wherein the controller is further configured to: segmenteach obstruction positioned on the structure as depicted by the image ofthe structure so that each obstruction is segmented from each planeincluded in the structure as depicted by the image, wherein eachobstruction is an obstruction positioned on the structure that is notincluded in any of the planes included in the structure.
 9. Theautomated structural feature analysis system of claim 7, wherein thecontroller is further configured to: locate each obstruction that ispositioned on the structure as depicted by the image as designated byeach line that encompasses each obstruction; and identify eachobstruction that is positioned on the structure to identify a structuralfeature of each obstruction that is positioned on the structure, whereinthe structural feature is a feature provided by each obstruction to thestructure.
 10. The automated structural feature analysis system of claim8, wherein the controller is further configured to: generate the sketchimage of the structure that is displayed to the user that depicts eachsegmented plane and each segmented obstruction included in the structurebased on the segmentation of each plane and each obstruction relative tothe distance between the 3D device and each point of the 3D beampositioned on the structure, the distance between each point on theimage, and the pitch of each plane.
 11. A method for automaticallygenerating and analyzing structural features of a structure, comprising:emitting a volume scanning Light Detection and Ranging (3D) beam thatscans a structure to generate 3D data that is associated with a distancebetween the 3D device and each end point of the 3D beam positioned onthe structure; capturing an image of the structure to generate imagedata associated with the structure as depicted by the image of thestructure; fusing the 3D data of the structure generated by a 3D devicewith the image data of the structure generated by an imaging device todetermine the distance between the 3D device and each end point of the3D beam positioned on the structure and determine a distance betweeneach point on the image.
 12. The method of claim 11, further comprising:generating a sketch image of the structure that is displayed to the userthat depicts the structure based on the distance between the 3D deviceand each point of the 3D beam positioned on the structure and thedistance between each point on the image, determining a pitch of eachplane included in the structure based on the distance between the 3Ddevice and each end point of the 3D beam positioned on the structure andthe distance between each point on the image, wherein each planeincluded in the structure is a portion of the structure that ispositioned at a different pitch relative to each other plane included inthe structure.
 13. The method of claim 12, further comprising:generating a point cloud map of the structure based on the fused 3D dataand the image data that includes a plurality of cloud points that isarranged to depict a three-dimensional (3D) representation of thestructure, wherein the point cloud map provides the distance between the3D device and each cloud point included in the point cloud map and thedistance between each cloud point included in the point cloud map. 14.The method of claim 13, further comprising: determining the pitch ofeach plane included in the structure based on the distance between the3D device and each cloud point included in the point cloud map and thedistance between each cloud point included in the point cloud map. 15.The method of claim 14, further comprising: segmenting each planeincluded in the structure as depicted by the image of the structure sothat each plane is segmented from each other plane included in thestructure as depicted by the image based on the fused 3D data and imagedata depicted in the point cloud map, wherein each point depicted by theimage corresponds to a cloud point on the point cloud map that providesthe distance between the 3D device and each cloud point and the distancebetween each cloud point.
 16. The method of claim 15, furthercomprising: determining a plurality of dimensions associated with eachplane and a pitch associated with each plane and a plurality ofdistances between each plane and each other plane based on thesegmentation of each plane included in the structure as depicted by theimage of the structure.
 17. The method of claim 16, further comprising:locating each element that formulates each plane included in thestructure as depicted by the image as designated by each line thatencompasses each plane; identifying each element that formulates eachplane included in the structure to identify a structural feature of eachelement of each plane, wherein the structural feature is a featureprovided by each element to the structure as formulated in eachcorresponding plane; and determine a distance of each element thatformulates each plane included in the structure, wherein the distance isa length dimension associated with each element.
 18. The method of claim17, further comprising: segmenting each obstruction positioned on thestructure as depicted by the image of the structure so that eachobstruction is segmented from each plane included in the structure asdepicted by the image, wherein each obstruction is an obstructionpositioned on the structure that is not included in any of the planesincluded in the structure.
 19. The method of claim 18, furthercomprising: locating each obstruction that is positioned on thestructure as depicted by the image as designated by each line thatencompasses each obstruction; and identifying each obstruction that ispositioned on the structure to identify a structural feature provided byeach obstruction to the structure.
 20. The method of claim 19, furthercomprising: generating the sketch image of the structure that isdisplayed to the user that depicts each segmented plane and eachsegmented obstruction included in the structure based on thesegmentation of each plane and each obstruction relative to the distancebetween the 3D device and each point of the 3D beam positioned on thestructure, the distance between each point on the image, and the pitchof each plane.